Sign synthesis from SignWriting notation using MPEG-4 standard, H- Anim and inverse kinematics techniques

A novel approach is presented for generating VRML animation sequences from Sign Language notation, based on MPEG-4 Face and Body Animation. Sign Language notation, in the well-known SignWriting system, is provided as input and initially converted to SWML (SignWriting Markup Language), an XML-based format that has recently been developed for the storage, indexing, and processing of SignWriting notation. Each basic sign, namely signbox, is then converted to a corresponding sequence of Body Animation Parameters (BAPs) of the MPEG-4 standard. Inverse Kinematics are also employed for synthesizing complex animation sequences (e.g. contacts). In addition, if a sign contains facial expressions, then these are converted to a sequence of MPEG-4 Facial Animation Parameters (FAPs), while exact synchronization between facial and body movements is achieved. These sequences, which can also be coded and/or reproduced by MPEG-4 BAP and FAP players, are then used to animate H-anim compliant VRML avatars, reproducing the exact gestures represented in the sign language notation. Envisaged applications include interactive information systems for the persons with hearing disabilities (Web, E-mail, info–kiosks), and automatic translation of written texts to sign language (e.g. for TV newscasts).